A 3D sensor captures 3D points on the surface of an object or a scene. These 3D point measurements are obtained in the coordinate system of the 3D sensor. Since the 3D sensor has a limited field of view, it is generally possible to observe only a section of the surface of the object surface from a single viewpoint. To capture a wider section of the surface, one displaces the 3D sensor, captures 3D points from the new viewpoint, measures and calculates the displacement of the 3D sensor before transforming the newly acquired 3D points in a common coordinate system. One of the challenges in 3D scanning, or more precisely in the field of 3D modeling based on 3D scanning, consists in performing this task as efficiently as possible while keeping an adequate level of accuracy for a targeted application.
3D modeling applications are numerous and spread from the industrial scan for design, part inspection, reverse engineering to modeling systems adapted to the medical domain for the production of orthotics, prostheses or anthropometric documentation. 3D scanning and modeling contribute to several other domains including arts, media, archeology as well as sciences and technologies in general.
Typically, the spatial relationship between two sensor positions will encompass six degrees of freedom (DOF) including three translation parameters and three orientation parameters. In some cases, it is possible to constrain the motion of the 3D sensor to fewer parameters. For instance, the motion of the 3D sensor can be limited to a translation. Without any limitation, in the following the six DOF case is considered.
Different approaches have been developed and implemented for obtaining the spatial relationship between two 3D sensor viewpoints. Several types of positioning devices have been developed and are available commercially.
Among them, mechanical arms allow one to attach the 3D sensor to its end effector. The mechanical arm system will provide the parameters of the spatial relationship between its end effector and a fixed coordinate system. The size, ease of usability, portability, and accuracy are some features that will affect the cost.
Optical tracking devices have better flexibility and a larger working volume since there is no need for a mechanical structure to be attached to the 3D sensor. In some cases, active or passive targets are affixed to the 3D sensor and the targets are tracked by the tracking system. However, this imposes that a minimum subset of targets be visible by the tracking system. These optical tracking systems may be active trackers or photogrammetric systems.
In the optical device category, it is also possible to affix one or more optical sensors (cameras) to the 3D sensor. These cameras will track active or passive targets set in the environment. The cameras track affixed features in the environment as opposed to tracking the 3D sensor itself.
Other types of positioning devices exist and they include electromagnetic, acoustic and inertial systems. Inertial systems do not provide a spatial relationship with respect to a fixed reference; they provide relative motion parameters of the device.
It is also known in the art to combine modalities. For instance, one can exploit an inertial system that is combined with an optical system.
Some systems are said to be auto-referenced in the sense that they do not need additional external hardware to measure their position in space. For instance the Handyscan™ technology by Creaform™ exploits the cameras integrated in the 3D sensor for both obtaining the sets of 3D points and measuring the position of targets affixed to the scene. These targets are designed to be extracted with high accuracy in the images. Moreover, the 3D sensors can learn the set of target positions during scanning and provide a level of accuracy that is comparable to that of costly positioning devices. Nevertheless, one will need to affix targets on the scene with a density sufficient to ensure that the system will always observe at least three target points at once, three being the minimum number of targets required to estimate a six DOF spatial relationship.
One can further improve flexibility of auto-referenced systems by eliminating the need to affix targets in the scene. One way to do that, exploits the measured geometry of the object with a full-field 3D sensor. Indeed, when the object is rigid (for at least the duration of the scan) and the displacement between each viewpoint is small with a large field of view overlap, the sets of 3D patches can be virtually moved in space to find the best fitting arrangement. Once the positions and orientations of the surface patches are calculated, one obtains the 3D sensor spatial relationship in a coordinate system that may be attached to one selected patch, for instance. Well known algorithms such as the Iterative Closest Point (ICP) and its numerous variants have been proposed for this type of pose refinement.
Some approaches have also been proposed for coarse registration and are applied when one cannot assume a small displacement between two 3D sensor positions. Coarse registration may rely on geometric feature recognition. The main limitation of these approaches exploiting the object geometry to calculate the sensor spatial relationship is the object geometry. In fact, it is assumed that the geometry is complex enough and contains no symmetry. Otherwise the positioning will be prone to error. For example, on a cylinder or a sphere, it is not possible to reliably calculate the six degrees of freedom. Furthermore, it is difficult to ensure a given level of accuracy which is crucial in many metrology applications. Moreover, for real-time modeling applications, coarse registration may become intractable for larger or more complex objects.
It is possible to combine texture with geometry information. Texture pixels or some feature points can be extracted and matched between viewpoints to improve the calculated sensor positions. Nevertheless, relying on these uncontrolled features may not be sufficient to maintain accuracy. It is also not simple to efficiently take advantage of both modalities using a relative weighting factor.